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Abstract Implementing evidence-based practices (EBPs) 
for children with autism is challenging for teachers because 
these practices are often complex, requiring significant 
training and resources that are not available in most school 
settings. This brief investigation was designed to identify 
areas of strength and difficulty for teachers implementing 
one such EBP, pivotal response training (PRT). Observa- 
tional data were gathered from 41 teachers participating in 
two separate investigations involving PRT. Despite dif- 
ferences in training procedures, teachers demonstrated 
similarities in areas of strength (clear opportunities/ 
instruction and child choice) and difficulty (turn taking and 
multiple cues). These findings suggest next steps toward 
systematic adaptation of PRT for classroom use. The 
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research may serve as a model for the process of adapting 
EBPs for practice settings. 
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Introduction 

Serving students with autism spectrum disorders (ASDs) 
poses a challenge to public schools because very few 
interventions have been developed for and systematically 
tested in school settings. Most evidence-based practices 
(EBPs) for children with ASD are complex, requiring 
specific training or resources that may not be available in 
most schools. Many teachers report using EBPs, but also 
report modifying them for use in the classroom (Stahmer 
et al. 2011; Stahmer et al. 2005). Teachers report com- 
bining and adapting EBPs from various training protocols 
to fit their own teaching preferences and the perceived 
needs of their students. Research in other areas suggests 
that the positive outcomes demonstrated in research set- 
tings may not be maintained when programs are modified 
in this way (Weisz et al. 1995). 

One EBP for students with ASD that is used in class- 
rooms is pivotal response training (PRT; Koegel et al. 
1989). PRT is a naturalistic intervention, based on the 
principles of applied behavior analysis, which is soundly 
supported in the scientific literature (National Autism 
Center 2009; Humphries 2003; Lord and McGee 2001; 
Wilczynski et al. 2011). A recent review listed PRT as one 
of 24 EBPs with evidence of efficacy for teaching students 
with ASD (Odom et al. 2010). 

Both comprehensive intervention packages and eclectic 
educational programs often include PRT as a teaching 
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technique (Arick et al. 2003; Stahmer et al. 2011; Stahmer 
and Ingersoll 2004), and in one survey most teachers 
reported using PRT (Stahmer 2007). However, it is unclear 
how teachers use PRT. It’s not known whether (1) the 
adaptations teachers make to PRT; or (2) the combination 
of PRT with other methods, as commonly occurs in com- 
prehensive programs, compromises its effectiveness. 

Evaluating teacher implementation of PRT and identi- 
fying challenges to its use in classrooms is an important 
first step in the larger effort to improve the quality of 
educational services for children with autism. Improved 
training protocols, additional materials, or adapted proce- 
dures may be needed to maximize the effectiveness of PRT 
in classroom settings. The purpose of this brief investiga- 
tion was to identify areas of strength and difficulty for 
teachers implementing PRT as an initial step toward 
translation of PRT for classroom use. 


Method 

Video observations of two groups of teachers using PRT in 
their classrooms were analyzed. One group of teachers was 
trained by a researcher as part of an efficacy trial (Research 
Trained; RT). The second group was trained by clinical 
trainers as part of an effectiveness trial (Clinically Trained; 
CT). Though training and observation procedures (see 
below) were not consistent across the studies, the two 
groups provide a preliminary view of fidelity of imple- 
mentation of PRT in school settings. 

Participants and Training 

Group 1 (RT) 

Participants included 19 teachers working in preschool- 
2nd grade special education classrooms serving children 
with ASD in Southern California. Teachers participating in 
this study were part of a larger study examining the effi- 
cacy of training classroom teachers in PRT procedures 
(Suhrheinrich 2011). All teachers were female. Fifty-three 
percent held Masters degrees. They had an average of 
6.5 years of experience teaching special education 
(.25-30 years) and 6.2 years of experience teaching chil- 
dren with ASD (.25-30 years). Participants attended a 6-h 
workshop on PRT that incorporated didactic instruction, 
modeling, and a manual created for clinical training by 
PRT developers (Koegel et al. 1989). The manual was 
slightly adapted to include examples of teachers, rather 
than parents, implementing the PRT components. All 
training was conducted by the principal investigator, who 
completed graduate level training in PRT with one of the 
program developers (L. Schreibman). After the workshop. 


teachers received feedback during two individual coaching 
sessions in their classrooms, which occurred weekly 
(approximately) after the workshop was completed. A 
complete description of the training received by RT 
teachers is available in Suhrheinrich (2011). 

Group 2 ( CT) 

Participants included 22 teachers working in K- 2nd grade 
ASD classrooms in an urban school district in Philadelphia, 
PA (Mandell et al. 20 13). These teachers were part of a 
larger study examining the implementation of EBPs in 
community programs. Of the 49 teachers participating in 
the larger study, these 22 were chosen for the current study 
because the dates of observation and coaching made their 
training most comparable with the Southern California 
group. All teachers were female. Sixty-four percent held 
Masters degrees. They had an average of 11.4 years of 
experience teaching special education (3-38 years) and an 
average of 6.5 years of experience teaching children with 
ASD (1-24 years). 

Clinically trained teachers received training in a com- 
prehensive instructional program for children with ASD 
(Strategies for Teaching based on Autism Research, STAR; 
Arick et al. 2003), which includes PRT, discrete trial 
training and teaching within functional routines. As part of 
their training, teachers and classroom staff participated in 
6 h of training focused on PRT strategies exclusively, 
based on the STAR manual developed for teacher use 
(Arick et al. 2004). Initial training was conducted by one of 
the authors of the STAR program, who completed graduate 
level training in PRT with one of the program developers 
(L. Schreibman) and followed a traditional PRT training 
format. After participants attended the workshop, coaching 
was provided on a monthly basis for the entire classroom 
intervention program; on average, teachers received 3.3 h 
of coaching per month (range = 0-5.9 h per month). 
Coaching was provided by local coaches with MA and 
doctoral level training in behavioral strategies, whom the 
STAR program developers trained in PRT. There is no 
estimate of the amount of coaching specifically for PRT. A 
complete description of the training received by CT 
teachers is available in Mandell et al. (2013). 

Measures 

Video Observations 

Research trained teachers were filmed during weekly 
coaching sessions following the group training workshop. 
Data from the two coaching sessions following training 
were averaged. One RT teacher had only one observation 
due to scheduling difficulties. CT teachers were scheduled 
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to be filmed monthly using PRT. Data for CT teachers with 
at least one PRT segment filmed during the 2 months fol- 
lowing PRT training were used in the current analyses. If 
two segments were available ( n — 7), the average was used 
in all analyses. For both groups, a trained research assistant 
filmed the teacher conducting PRT in the classroom. 

For both groups, visits were scheduled with the teacher 
beforehand; all teachers were aware when videotaping 
would occur. In both groups, the teacher selected the 
materials for the teaching interaction and worked one-on- 
one with a student with ASD in their classroom using PRT. 

Fidelity of Implementation Coding 

Video observations were coded to assess the teachers’ 
fidelity of implementation of PRT. Research assistants, 
who were blind to the research hypotheses and teachers’ 
training group or experience level, were trained to code the 
video samples using a set of behavioral definitions for each 
component of PRT (see Table 1 for abbreviated procedures 
and definitions). 

Coders observed a 10-min video clip in which the tea- 
cher used PRT with an individual student. Coders rated the 


use of each component of PRT on 1-5 Likert scale after 
viewing the entire clip. A score of one indicated the teacher 
did not use the strategy during the session or never 
implemented it correctly. A score of five indicated the 
teacher implemented the component competently 
throughout the segment. In order to meet fidelity of 
implementation on a particular component of PRT, teach- 
ers needed to receive a score of 4 (implements the com- 
ponent competently a majority of the time, but misses some 
opportunities) or 5 (implements the component compe- 
tently throughout the session). The Likert coding system 
was developed as part of a larger effort to adapt fidelity of 
implementation assessment procedures for feasibility in 
clinical settings. The development process included sys- 
tematic comparison of trial-by-trial and interval based 
coding of occurrence/non-occurrence of each component 
of PRT to the adapted Likert scale model. A regression 
analysis indicted a significant correlation between the 
Likert coding and the trial by trial method (R 2 = .72). 
Overall pass/fail agreement for PRT fidelity occurred in 
90 % of cases when passing on the trial by trial coding 
was defined as correct implementation in 80 % of trials 
and passing on the Likert rating was defined as a rating of 4 


Table 1 Summary of PRT Components 
PRT component Definition 


Gains attention Teacher must have student’s attention before presenting an opportunity to respond 

Clear opportunity/ The question/opportunity to respond must be developmentally appropriate for the child and & appropriate to task 

instruction 


Maintenance tasks a Tasks that are easy must be interspersed with more difficult tasks (acquisition) 

Child choice (shared Teacher must follow the student’s choice of tasks, to a large extent and/or provide choices within tasks 

control) 


Turn taking (shared Teacher models appropriate behavior while maintaining control of the instructional materials in the context of a give 
control) and take interaction with the student 


Multiple cues 

Contingent 

consequences 

Direct reinforcement 
Reinforcement of 
attempts 3 
Coding criteria 
1 
2 

3 

4 

5 


Some instructions should involve cues that include multiple components (two or more aspects of the environment, 
stimuli or activity) 

Reinforcement must be contingent on the child’s behavior 

Reinforcement should be natural and directly related to the desired behavior 
Goal-directed attempts to respond are reinforced 

Definition 

Teacher does not implement throughout session 

Teacher implements occasionally, but misses majority of opportunities 
Teacher implements up to half the time hut misses many opportunities 
Teacher implements a majority of the time but misses some opportunities 
Teacher implements competently through the session 


Coding procedures 

Score each component based on your observation of the teacher-student interaction. Observe the session or selected video segment and take notes 
as needed to indicate correct/incorrect use of the components. Then, select one code that best represents the amount of time in which the teacher 
correctly implemented each component. 


a Not coded for fidelity of implementation in the current investigation 
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or 5. Coding involved direct computer entry while viewing 
the video using “The Observer Video-Pro” software 
(Noldus Information Technology, Inc.), a computerized 
system for collection, analysis and management of obser- 
vational data. 

Double scoring of 33 % of the video samples from the 
CT group and 27 % of the video samples of the RT group 
estimated 86 % inter-rater agreement for videos of CT 
teachers and 83 % for videos of RT teachers. 

Data Analysis 

The Likert rating for each PRT component was averaged 
across observations for each teacher (when multiple 
observations were available), in order to calculate a single 
1-5 score per component, per teacher. Scores on individual 
components across teachers in each group were then 
averaged to compare PRT implementation between RT and 
CT teachers. A mixed model ANOVA with PRT compo- 
nent as a within subject factor and RT versus CT group as a 
between subjects factor was used to examine differences in 
performance across both training groups and components. 

The percentage of teachers within each group who met 
fidelity on each component was calculated and examined 
visually to inform which components were easy or difficult 
for teachers to implement correctly and to examine 
potential differences based on type of training received. 
Based on the range of teacher fidelity, high fidelity was 
defined as 70 % or more of teachers within a group 
implementing the component correctly, moderate fidelity 
was defined as 40-69 % of teachers within a group 
implementing the component correctly, and low fidelity 
was defined as 39 % or fewer teachers within a group 
implementing the component correctly. These categories 
may be considered tools for descriptive analysis of the 
results. 


Results 

The fidelity of implementation of each component was 
examined across RT and CT groups (see Table 2). 

There was no statistically significant association 
between teachers’ fidelity of implementation of PRT 
(either individual components or overall) and years of 
experience teaching special education or teaching children 
with ASD. RT teachers, as a group, met the fidelity of 
implementation criteria for all of the PRT components 
except turn taking and multiple cues. As a group, CT 
teachers did not meet fidelity of implementation criteria for 
any of the components. CT teachers implemented the 
components child choice, clear instructions, and gains 
attention at the highest levels of fidelity out of all the 


components, with a majority of teachers passing (receiving 
a score of 4 or 5), but average scores across all teachers 
were not in the passing range. 

Training group was associated with teachers’ fidelity of 
implementation of PRT (p < .01). Post hoc t-tests revealed 
that RT teachers implemented all PRT components except 
turn taking and multiple cues with significantly higher 
fidelity than CT teachers (p < 05). Fidelity of implemen- 
tation for turn taking and multiple cues components was 
similar across both groups. 

Teachers’ fidelity of implementation varied significantly 
by component (p < .01). Results from post hoc paired 
sample t-tests (corrected by Holm’s method) revealed that 
three components, gains attention, clear instructions, and 
child choice, were implemented with significantly higher 
fidelity than all other components (p < .05). The compo- 
nents of turn taking and multiple cues were implemented 
with significantly lower fidelity than all other components 

(p < -OD- 

Examination of the percentage of teachers passing 
fidelity on each component also reveals similar areas of 
strength and difficulty across groups (see Figure 1). 

Providing a clear opportunity/instruction and incorpo- 
rating child choice were areas of strength, or high fidelity, 
with over 70 % of teachers in each group meeting fidelity 
of implementation criterion in this area (clear opportunity, 
RT = 100 %; CT = 77 %; child choice, RT = 100 %, 
CT = 73 %). Two components were implemented with 
low fidelity. Fewer than 40 % of teachers in each group 
implemented the turn taking and multiple cues components 
with fidelity (turn taking, RT = 26 %, CT = 36 %; mul- 
tiple cues, RT = 0 %, CT = 0 %). The percentage of 
teachers correctly implementing the remaining components 
varied by group. For gaining attention (RT = 95 %, 
CT = 50 %), providing contingent consequences 
(RT = 78 %, CT = 45 %) and providing direct rein- 
forcement (RT = 79 %, CT = 41 %), the RT teachers 
were at high fidelity while the CT teachers were at mod- 
erate fidelity. 

Discussion 

This initial investigation indicates similarity across two 
different groups of teachers in their implementation of PRT 
components after receiving training based on two widely 
available manuals developed for clinical use. These data 
demonstrate that teachers are skilled at using some com- 
ponents of PRT in classroom settings (child attention, clear 
opportunity/instruction, child choice). For other compo- 
nents, teachers may benefit from additional resources or 
training (contingent consequences, direct reinforcement). 
Two components appeared to be most problematic for 
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Table 2 Summary of observational findings by group 


Mean fidelity of implementation score 


Researcher trained Clinically trained 

(n = 19) (n = 22) 


PRT component 

M (SD) 

M (SD) 

Overall fidelity 

4.23 (.27) 

2.98 (.50) 

Gains attention 

4.86 (.36) 

3.46 (.67) 

Clear opportunity/ 

4.77 (.39) 

3.77 (.55) 

instruction 

Child choice 

4.74 (.51) 

3.85 (.72) 

Turn taking 

2.74 (1.4) 

3.05 (.99) 

Multiple cues 

1.33 (1.26) 

0.88 (1.13) 

Contingent 

4.35 (.69) 

3.30 (.84) 

consequences 

Direct reinforcement 

4.47 (.69) 

3.20 (.89) 


Research Trained 
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Fig. 1 Percent of teachers implementing PRT components at or 
above fidelity criteria 


teachers, as evidenced by poor implementation or exclu- 
sion during their use of PRT (multiple cues, turn taking). 
Based on these preliminary data we provide several rec- 
ommendations for possible next steps for translation of 
PRT to the classroom that include improved training, and 
further research to examine modifications. 

First, teachers in both groups had moderate difficulty pro- 
viding consequences that were contingent upon student 
behavior and providing direct reinforcement. Previous 
research suggests these components may be critical to student 
learning (Koegel et al. 1987; Koegel and Williams 1980; 
McGee et al. 1985; Schreibman and Koegel 2005; Williams 
et al. 1981). Achieving accurate implementation of these 
components in schools may require additional resources and 
strategies specific for classroom use. For example, to improve 
teachers’ use of contingent consequences, resources may 


include information on why providing contingent and direct 
consequences is important (based on supporting research) and 
how teachers can improve their use of contingent and direct 
consequences with real-world examples and activity sugges- 
tions. In addition, creative methods of providing direct rein- 
forcement, for example through token systems or other 
commonly used classroom procedures, may be helpful. 
Improved training procedures, such as providing more 
opportunity for behavioral rehearsal and coaching with feed- 
back on the components, may also improve teacher imple- 
mentation (Suhrheinrich 201 1). 

Teachers in both groups demonstrated difficulty imple- 
menting turn taking and multiple cues procedures. Tradi- 
tionally, both of the components have been included in PRT 
training (turn taking as part of shared control; Koegel et al. 
1989); however some recent publications on PRT have not 
included these steps explicitly. It is possible that this drift has 
occurred due to provider difficulty implementing these steps, 
but the effect of removing these specific components, 
described in the original manual and present in the early 
research, has not been empirically documented. Additionally, 
recent large-scale reviews of intervention practices for indi- 
viduals with ASD have included these components. For 
example, responding to multiple cues is described as a pivotal 
area of PRT in the National Standards Project (National 
Autism Center 2009) and a component of the intervention in 
National Professional Development Center (National Pro- 
fessional Development Center on Autism Spectrum Disoders 
201 1). Next steps in translating PRT into community practice 
may require evaluating how modification or removal of these 
components affects overall efficacy of PRT for positive child 
outcomes. First, the specific factors that make each compo- 
nent difficult for teachers to implement should be explored. 
For example, the turn taking component may be difficult for 
teachers because they typically demonstrate new skills for 
students in an instructional manner, but do not model new 
skills by taking turns with students once an activity has begun. 
Identifying how providing a demonstration (rather than 
modeling in the context of a turn) affects student learning 
would inform the process of possible modification of turn 
taking for future classroom use. Additional research on the 
multiple cues component of PRT may involve exploration of 
how broadening attention can be achieved through other 
teaching methods and the investigation of the developmental 
appropriateness of conditional discriminations for some stu- 
dents with ASD. A recent report indicates typically develop- 
ing children consistently respond to simultaneous multiple 
cues at 36 month on average (Reed et al. 2012). These 
findings may inform how developmental readiness of children 
with ASD impacts the added benefit of this component. Each 
of the components found to be difficult to implement should be 
deconstructed and systematically evaluated to best inform the 
process of translating PRT to the classroom. 
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Teachers in the RT group had higher fidelity of imple- 
mentation overall than teachers in the CT group. There are 
at least two reasons why this might be the case. First, 
although teachers in both groups attended a 6 h workshop 
focused exclusively on PRT, teachers in the RT group were 
taught only PRT while teachers in the CT group were also 
taught a complete classroom curriculum during additional 
group workshops, which included two other intervention 
strategies. Anecdotally, CT teachers reported that the 
coaches put a strong emphasis on mastery of other methods 
and less emphasis on PRT, which may have impacted 
teacher mastery of the intervention. Additionally, although 
the workshop presenters for both groups were Master’s 
level trainers with expertise in PRT trained at the same 
graduate institution, there may have been more subtle 
differences in the trainers or workshop presentations that 
led to differences in teacher implementation. Having a 
more detailed understanding of the content of training and 
coaching in future research will provide needed informa- 
tion regarding appropriate training methodology for trans- 
lation of EBP to community settings. 

Overall, the results of this investigation indicate PRT is a 
viable intervention for teacher use in classroom settings and 
add to the small body of literature describing teacher fidelity of 
implementation of PRT or similar techniques (e.g., Jones et al. 
2006; Smith and Camarata 1999; Suhrheinrich et al. 2007). 
There was consistent evidence across participant groups to 
indicate areas for possible adaptation (i.e. turn taking, multiple 
cues). Additionally, this research may serve as a model for the 
process of adapting EBPs for clinical settings more generally. 
Innovative efforts toward dissemination of EBPs have shifted 
from the traditional, unidirectional models of translating 
research into practice toward a more reciprocal, interactive 
effort between researchers and practitioners (Bondy and 
Brownell 2004; Meline and Paradiso 2003; Weisz et al. 2004). 
Providing more opportunity for intervention in applied set- 
tings to inform research is a likely first step toward effective 
translation of EBPs. Additionally, the process of gathering 
information about what works in educational settings should 
improve the quality of resources resulting from systematic 
scientific adaptation of interventions. 
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